Smoke detection method and apparatus

ABSTRACT

Embodiments of this disclosure provide a smoke detection method and apparatus. The apparatus includes a processor configured to detect foreground blobs in images of a plurality of frames in a video; determine motion areas of detected foreground blobs; determine a first degree of overlapping of the foreground blobs in images of at least two frames, and/or determine a second degree of overlapping of the foreground blobs and motion areas to which the foreground blobs correspond; determine interfering foreground blobs according to the first degree of overlapping and/or the second degree of overlapping; and extract features from motion areas of remaining foreground blobs with the interfering foreground blobs being removed, and detect smoke according to the features. With the embodiments of this disclosure, accuracy of smoke detection may be increased, and false detection resulted from such interfering objects as water blobs, or light spots, etc., may be avoided.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and hereby claims priority to ChineseApplication No. 201811568382.2, filed Dec. 21, 2018, in the StateIntellectual Property Office of China, the disclosure of which isincorporated herein by reference.

FIELD

This disclosure relates to the field of image processing, and inparticular to a smoke detection method and apparatus.

BACKGROUND

Fires often cause great damage to people. Since smoke usually occursbefore the fire spreads, if smoke can be detected as soon as possible,it may effectively prevent the spread of fire. At present, smokedetection algorithms based on conventional computer vision and patternrecognition have achieved good detection results.

It should be noted that the above description of the background ismerely provided for clear and complete explanation of this disclosureand for easy understanding by those skilled in the art. And it shouldnot be understood that the above technical solution is known to thoseskilled in the art as it is described in the background of thisdisclosure.

SUMMARY

It was found by the inventors that if a fire occurs in a harsh outdoorenvironment such as rain or excessive light reflection, water blobs, orlight spots of light reflection may fall on a lens of a video monitor,which may result in false detection of smoke.

Embodiments of this disclosure provide a smoke detection method andapparatus and an image processing device to solve the problems existingin the related art, which may increase accuracy of smoke detection, andavoid false detection resulted from such interfering objects as waterblobs, or light spots, etc.

According to a first aspect of the embodiments of this disclosure, thereis provided a smoke detection apparatus, including a memory and aprocessor. According to an embodiment, the processor is configured todetect foreground blobs in images of a plurality of frames in a video;determine motion areas of detected foreground blobs; determine a firstdegree of overlapping of the foreground blobs in images of at least twoframes, and/or determine a second degree of overlapping of theforeground blobs and motion areas to which the foreground blobscorrespond; determine at least one interfering foreground blob accordingto the first degree of overlapping and/or the second degree ofoverlapping; and extract features from motion areas of remainingforeground blobs with the at least one interfering foreground blob beingremoved, and detect whether smoke exists in the video according to theextracted features.

According to a second aspect of the embodiments of this disclosure,there is provided a smoke detection method. The method, according to anembodiment, includes detecting foreground blobs in images of a pluralityof frames in a video; determining motion areas of detected foregroundblobs; where a first degree of overlapping of the foreground blobs inimages of at least two frames is determined, and/or a second degree ofoverlapping of the foreground blobs and motion areas to which theforeground blobs correspond is determined. The method includesdetermining at least one interfering foreground blob according to thefirst degree of overlapping and/or the second degree of overlapping; andextracting features from motion areas of remaining foreground blobs withthe at least one interfering foreground blob being removed, where smokeis detected according to the extracted features.

According to a third aspect of the embodiments of this disclosure, thereis provided an image processing device, including the smoke detectionapparatus as described in the first aspect.

Advantages of the embodiments of this disclosure exist in that at leastone interfering foreground blob in the smoke detection are determinedand removed according to the degree of overlapping of the foregroundblobs in images of at least two frames and/or according to the degree ofoverlapping of the foreground blobs and the motion areas to which theforeground blobs correspond. Hence, the problems existing in the relatedart may be solved, accuracy of smoke detection may be increased, andfalse detection resulted from such interfering objects as water blobs,or light spots, etc., may be avoided.

With reference to the following description and drawings, the particularembodiments of this disclosure are disclosed in detail, and theprinciples of this disclosure and the manners of use are indicated. Itshould be understood that the scope of the embodiments of thisdisclosure is not limited thereto. The embodiments of this disclosurecontain many alternations, modifications and equivalents within thescope of the terms of the appended claims.

Features that are described and/or illustrated with respect to oneembodiment may be used in the same way or in a similar way in one ormore other embodiments and/or in combination with or instead of thefeatures of the other embodiments.

It should be emphasized that the term “comprises”, “comprising”,“includes” “including” when used in this specification is taken tospecify the presence of stated features, integers, steps/operations orcomponents but does not preclude the presence or addition of one or moreother features, integers, steps/operations, components or groupsthereof.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with referenceto the following drawings. The components in the drawings are notnecessarily to scale, emphasis instead being placed upon clearlyillustrating the principles of this disclosure. To facilitateillustrating and describing some parts of the disclosure, correspondingportions of the drawings may be exaggerated or reduced. Elements andfeatures depicted in one drawing or embodiment of the disclosure may becombined with elements and features depicted in one or more additionaldrawings or embodiments. Moreover, in the drawings, like referencenumerals designate corresponding parts throughout the several views andmay be used to designate like or similar parts in more than oneembodiment.

In the drawings:

FIG. 1 is a schematic diagram of the smoke detection method according toan embodiment;

FIG. 2 is a schematic diagram of the first degree of overlappingaccording to an embodiment;

FIG. 3 is schematic diagram of the second degree of overlappingaccording to an embodiment;

FIG. 4 is a flowchart of the smoke detection method according to anembodiment;

FIG. 5 is a schematic diagram of the smoke detection apparatus accordingto an embodiment; and

FIG. 6 is a schematic diagram of a hardware structure of the imageprocessing device according to an embodiment.

DETAILED DESCRIPTION

These and further aspects and features of the present disclosure will beapparent with reference to the following description and attacheddrawings. These embodiments are illustrative only, and not intended tolimit this disclosure. In order that those skilled in the art are ableto understand easily the principle and embodiments of this disclosure,the embodiments of this disclosure shall be described by taking areconstructed image of image compression processing as an example.However, it should be understood that the embodiments of this disclosureare not limited thereto, and other reconstructed images based on imagecompression processing are also covered by the scope of this disclosure.

In the embodiments of this disclosure, terms “first”, and “second”,etc., are used to differentiate different elements with respect tonames, and do not indicate spatial arrangement or temporal orders ofthese elements, and these elements should not be limited by these terms.Terms “and/or” include any one and all combinations of one or morerelevantly listed terms. Terms “contain”, “include” and “have” refer toexistence of stated features, elements, components, or assemblies, butdo not exclude existence or addition of one or more other features,elements, components, or assemblies.

In the embodiments of this disclosure, single forms “a”, and “the”,etc., include plural forms, and should be understood as “a kind of” or“a type of” in a broad sense, but should not defined as a meaning of“one”; and the term “the” should be understood as including both asingle form and a plural form, except specified otherwise. Furthermore,the term “according to” should be understood as “at least partiallyaccording to”, the term “based on” should be understood as “at leastpartially based on”, except specified otherwise.

The embodiments of this disclosure shall be described below withreference to the accompanying drawings.

Embodiment 1

Embodiment 1 provides a smoke detection method. FIG. 1 is a flowchart ofthe method. As shown in FIG. 1, the method includes:

operation 101: foreground blobs are detected in images of a plurality offrames in a video;

operation 102: motion areas of detected foreground blobs are determined;

operation 103: a first degree of overlapping of the foreground blobs inimages of at least two frames is determined, and/or a second degree ofoverlapping of the foreground blobs and motion areas to which theforeground blobs correspond is determined;

operation 104: interfering foreground blobs are determined according tothe first degree of overlapping and/or the second degree of overlapping;and

operation 105: features are extracted from motion areas of remainingforeground blobs with the interfering foreground blobs being removed,and smoke is detected according to the features.

In this embodiment, the interfering foreground blobs in the smokedetection are determined and removed according to the degree ofoverlapping of the foreground blobs in images of at least two framesand/or according to the degree of overlapping of the foreground blobsand the motion areas to which the foreground blobs correspond. Hence,the problems existing in the related art may be solved, accuracy ofsmoke detection may be increased, and false detection resulted from suchinterfering objects as water blobs, or light spots, etc., may beavoided.

In this embodiment, in operation 101, the video including images of aplurality of frames may be acquired by using devices such as a camera,and the video is decoded to obtain the images of a plurality of frames.A decoding method may refer to related art, for example, the obtainedvideo stream is subjected to entropy decoding, inverse quantization,inverse transform, and combined with the inter-frame prediction resultto obtain images of decoded frames.

In this embodiment, in operation 101, before detecting the foregroundblobs, in order to improve the accuracy of the detection result of theforeground blobs, the frame image may be pre-processed, such assharpening each frame image, etc., so as to enhance edges of the imageand grayscale hopping part, make the image more clear, eliminate orreduces noises, and extract the foreground blobs. The related art, suchas high-pass filtering or spatial domain differentiation, etc., may beused for the sharpening processing, and this embodiment is not limitedthereto.

In this embodiment, in operation 101, in detecting the foreground blobsaccording to the images of plurality of (N) frames, such algorithms asframe difference, background subtraction, and optical flow, etc., in therelated art may be used. For example, a background image modeling methodbased on the Gaussian Mixture Model (GMM) may be used to performbackground modeling on the images of a plurality of frames to obtain theforeground blocks and a background image; however, this disclosure isnot limited thereto.

In this embodiment, in operation 102, in order to analyze features ofsmoke during a motion process, it is needed to perform tracking analysison the foreground blobs suspected of smoke in a time sequence, that is,to track a motion process of each foreground blob in the images ofplurality of (N) frames, so as to obtain motion areas of the foregroundblobs in the images of a plurality of frames. An existing blob matchingalgorithm may be used as the tracking method, such as dividing the frameimage into different macroblobs, and performing search matching within aset range of adjacent frame images to find a best blob in the currentframe (N-th frame) similar to the foreground blob. The existing minimummean absolute difference (MAD) and minimum mean square error (MSE)algorithm, or the like, may be used for judging a similarity. Positionsof the foreground blobs in the frame images may be determined accordingto the above tracking algorithm, and the motion areas of the foregroundblobs are constituted by combining the above tracked positions in theimages of a plurality of frames.

In this embodiment, alternatively, in order to reduce an amount ofcalculation and improve detection precision, pre-screening may beperformed in operations 101-102, that is, detecting foreground blobssuspected of being smoke and motion areas of the foreground blobssuspected of being smoke in all foreground blobs and motion areas. Forexample, when shapes, contours, colors, etc., of the detected foregroundblobs do not conform to features of smoke, such foreground blobs neednot to be tracked, or when motion areas of the detected foreground blobsare moving downwards or completely motionless, that is, do not conformto motion features of smoke, processing in subsequent operations 103-105need not to be performed on such foreground blobs.

In this embodiment, in an outdoor environment, when rain occurs or lightreflection is relatively strong, water blobs or light spots may fall ona lens of a video monitor, and since such features as colors orcontours, etc., of the water blobs and the light spots are similar tosmoke, it is possible that false detection will be resulted in.Considering that motion modes of such interfering objects as water blobsor light spots are different from that of smoke, for example, comparedwith the above-mentioned interfering objects, moving speed of the smokeis faster, the smoke does not stay in one position for a long time, andthe area of a motion area is relatively large, while such interferingobjects as water blobs or light spots may stay at a position in thevideo image for a relatively long time, and the area of motion areas arerelatively small. This embodiment proposes a method for determining andremoving interfering foreground blobs from foreground blobs suspected ofbeing smoke according to degrees of overlapping of foreground blobs inimages of at least two frames and/or according to degrees of overlappingof foreground blobs and motion areas to which the foreground blobscorrespond, which shall be described below in detail in operations 103and 104.

In this embodiment, in one implementation of operations 103-104, theinterfering foreground blobs may be determined according to theforeground blobs (or may also be foreground blobs suspected of beingsmoke, which shall not be differentiated in the following description)detected in operation 101.

In this implementation, after the foreground blobs are detected inoperation 101, the interfering foreground blobs are determined duringthe foreground blob tracking process in operation 102. In calculating afirst degree of overlapping a foreground blob, a first overlapping areaof the foreground blob in images of a first number of frames iscalculated, a first area of the foreground blob is calculated, and aratio of the first overlapping area to the first area is taken as thefirst degree of overlapping of the foreground block. A method forcalculating first degrees of overlapping of other foreground blobs isidentical to the method for calculating the first degree of overlappingof the foreground blob.

In this implementation, in order to calculate the overlapping area, thefirst number needs to be greater than or equal to 2. In order to improvethe detection precision, the first number of frames may be adjacentframes, and the adjacent frames denote frames adjacent to one frame inmultiple frames in a certain range. The one frame may be the currentframe (the N-th frame), or may be a frame preceding the current frame(an (N−1)-th frame), or may be an (N−i)-th frame (i being greater than1); however, this embodiment is not limited thereto. Alternatively, theadjacent frames are consecutive frames. And the first area may be anarea of the one foreground blob in an image of any frame in the image ofN frames, such as the (N−1)-th frame, or the N-th frame; however, thisembodiment is not limited thereto.

For example, when the first number is 2, the first number of frames maybe the current frame in the images of a plurality of frames of thevideo, i.e. the N-th frame, and a frame preceding the current frame,i.e. the (N−1)-th frame. FIG. 2 is a schematic diagram of the firstdegree of overlapping. As shown in FIG. 2, the foreground blobs detectedin the N-th frame are A, B, C and D, and the foreground blobs detectedin the (N−1)-th frame are A′, B′, C′ and D′. In the foreground blobtracking process, A-A′ are the same foreground, B-B′ are the sameforeground, C-C′ are the same foreground, and D-D′ are the sameforeground. For the foreground blob A-A′, a first degree of overlappingthereof may be calculated by using formula (1) below:

$\begin{matrix}{{{IoU}_{1} = \frac{{Blob}_{N - 1}\bigcap{Blob}_{N}}{{Blob}_{N - 1}}};} & {{formula}\mspace{14mu}(1)}\end{matrix}$

where, IoU₁ denotes the first degree of overlapping, Blob_(N-1) and BlobN may both denote the first area, and Blob_(N-1)∩Blob_(N) denotes afirst area of overlapping of the foreground blob in the N-th frame andthe (N−1)-th frame. How to calculate the first degree of overlapping isdescribed above by taking that the first number is 2 and the foregroundblob is the foreground blob A-A′ as an example. Methods for calculatingdegrees of overlapping of the foreground blobs B-B′, C-C′ and D-D′ aresimilar to formula (1). And furthermore, the first number may be greaterthan 2, and the first area of overlapping may also be overlapped partsof the foreground blob in an image of more than three frames, whichshall not be enumerated herein any further.

In this implementation, in comparison with smoke, as water blobs orlight spots will not move substantially in adjacent frames, in otherwords, the first overlapping area of the interfering foreground block isclose to the first area, and the first degree of overlapping isrelatively large, a first threshold may be predetermined, and foregroundblocks with first degrees of overlap being greater than or equal to thefirst threshold are determined as the interfering foreground blocks. Thefirst threshold may be determined empirically, and this embodiment isnot limited thereto.

In this embodiment, in another implementation of operations 103-104, theinterfering foreground blocks may be determined according to theforeground blobs (or may also be foreground blobs suspected of beingsmoke, which shall not be differentiated in the following description)detected in operation 101 and the motion areas (or may also be motionareas suspected of being smoke, which shall not be differentiated in thefollowing description) of the foreground blocks detected in operation102.

In this implementation, after the end of the tracking in operation 102,the interfering foreground blocks may be determined according to theforeground blobs detected in operation 101 and the motion areas of theforeground blobs detected in operation 102. In calculating a seconddegree of overlapping of a foreground blob, a second area of overlappingof an area of the foreground blob and a second area of a motion area ofthe foreground block is calculated, and a ratio of the second area ofoverlapping to the second area as the second degree of overlapping. Amethod for calculating second degrees of overlapping of other foregroundblobs is identical to the method for calculating the second degree ofoverlapping of the foreground blob.

In this implementation, reference may be made to the above first areafor a method for calculating the area of the foreground blob. The secondarea may be an area of the motion area detected in the above N frames,or may be an area of a motion area detected in a second number of framesat rear positions in the N frames, in which case the area of theforeground blob may be an area in an image of one frame in the secondnumber of frames; however, this embodiment is not limited thereto. Thesecond number is greater than or equal to the first number, and thesecond number of frames may be adjacent frames, and optionally, they maybe consecutive frames.

For example, the area of the foreground block is the area of the N-thframe, and the area of the motion area is the area of the motion area ofthe foreground blob in the N frames. FIG. 3 is a schematic diagram ofthe second degree of overlapping. As shown in FIG. 3, the foregroundblobs detected in the N-th frame are A and B, and after the foregroundblob tracking process, a motion area of A in the N frames is A″, and amotion area of B in the N frames is B″. For the foreground block A, thesecond degree of overlapping thereof may be calculated by using formula(2) below:

$\begin{matrix}{{{IoU}_{2} = \frac{{Blob}_{A}\bigcap{Blob}_{A^{''}}}{{Blob}_{A^{''}}}};} & {{formula}\mspace{14mu}(2)}\end{matrix}$

where, IoU₂ denotes the second degree of overlapping, Blob_(A) denotesthe area of the foreground block, Blob_(A″) denotes the area of themotion area of the foreground block, and Blob_(A)∩Blob_(A″) denotes thesecond area of overlapping. How to calculate the second degree ofoverlapping is described above by taking the foreground block A as anexample, and a method for calculating the second degree of overlappingof the foreground block B is similar to formula (2), which shall not beenumerated herein any further.

In this implementation, in comparison with smoke, as water blobs orlight spots will not move substantially in adjacent frames, in otherwords, the area of the motion area of the interfering foreground blob isrelatively small and close to the area of the foreground blob, and thesecond degree of overlapping is relatively large, a second threshold maybe predetermined, and foreground blocks with second degrees of overlapbeing greater than or equal to the second threshold are determined asthe interfering foreground blocks. The second threshold may bedetermined empirically, and this embodiment is not limited thereto.

In this embodiment, in another implementation of operations 103-104, theabove first degree of overlapping and second degree of overlapping maybe respectively calculated, and the interfering foreground blocks aredetermined and removed according to the first degree of overlapping andsecond degree of overlapping, which shall not be described herein anyfurther.

In this embodiment, in operation 105, the features are extracted fromthe motion areas of the remaining foreground blobs (or remainingforeground blobs suspected of being smoke) with the interferingforeground blobs being removed. For example, the features may includesaturation-related information, a grayscale variance value, and agradient grayscale mean value, and the like. Whether there exists smokeis determined according to the features. Reference may be made to therelated art for a method for extracting the above features, which shallnot be enumerated herein any further.

For example, a saturation of smoke is generally low, and when anextracted feature is a saturation mean value, whether the saturationmean value is greater than or equal to a third threshold is judged, andwhen it is greater than or equal to the third threshold, it isdetermined that there exists no smoke.

For example, a texture of smoke is generally low, and when an extractedfeature is a grayscale variance, whether the grayscale variance isgreater than or equal to a fourth threshold is judged, and when it isgreater than or equal to the fourth threshold, it is determined thatthere exists no smoke.

For example, smoke generally has a feature of diffuse divergence, andwhen an extracted feature is a grayscale mean value, whether adifference between the grayscale mean value and a background grayscalemean value is less than or equal to a fifth threshold is judged, andwhen it is less than or equal to the fifth threshold, it is determinedthat there exists no smoke.

How to judge whether there exists smoke is schematically describedabove; however, this disclosure is not limited thereto, for example,other features may be used for judgment, such as a gradient direction,or the like. Furthermore, one or more of the above implementations maybe employed. For example, only one of the implementations may be used,or at least two of the above implementations may be used. Moreover, anorder of the above judgment is not limited, and in practical use, adetailed detection scheme may be determined according to an actualsituation.

FIG. 4 is a flowchart of the smoke detection method of this embodiment.As shown in FIG. 4, the method includes:

operation 401: video data is acquired, and the video data is decoded toobtain images of a plurality of frames;

operation 402: the video data is preprocessed (optional);

operation 403: foreground blobs are detected in the images of aplurality of frames;

operation 404: motion areas of detected foreground blobs are determined;

operation 405: interfering foreground blobs are removed according to theforeground blobs and the motion areas;

operation 406: features are extracted in motion areas of remainingforeground blobs; and

operation 407: smoke is detected according to the extracted features.

In this embodiment, reference may be made to above-described operations101-102 for implementations of operations 401-404, and reference may bemade to above-described operation 105 for implementation of operations406-407, which shall not be described herein any further.

In this embodiment, in operation 405, the first degree of overlappingand/or the second degree of overlapping may be calculated according tothe foreground blobs and the motion areas, and the interferingforeground blobs (water blobs or light spots) may be determinedaccording to the first degree of overlapping and/or the second degree ofoverlapping, and reference may be made to operations 103-104 forimplementations thereof, which shall not be described herein anyfurther. Furthermore, in operation 405, it is included to use othermethods to remove other types of interfering foreground blobs, such asremoving mobile interfering foreground blobs or static interferingforeground blobs, such as vehicles and people, etc. Reference may bemade to the related art for a method for determining the mobileinterfering foreground blobs or static interfering foreground blobs,which shall not be described herein any further. Hence, with operation405, some of interfering foreground blobs in the detected foregroundblobs are removed in advance, which may not only improve accuracy ofsmoke detection, but also improve speed of smoke detection.

With the above embodiment, the interfering foreground blobs in the smokedetection are determined and removed according to the degree ofoverlapping of the foreground blobs in images of at least two framesand/or according to the degree of overlapping of the foreground blobsand the motion areas to which the foreground blobs correspond. Hence,the problems existed in the related art may be solved, accuracy of smokedetection may be increased, and false detection resulted from suchinterfering objects as water blobs, or light spots, etc., may beavoided.

Embodiment 2

Embodiment 2 provides a smoke detection apparatus. As principles of theapparatus for solving problems are similar to that of the method inEmbodiment 1, reference may be made to the implementation of the methodin Embodiment 1 for implementation of the apparatus, with identicalcontents being not going to be described herein any further.

FIG. 5 is a schematic diagram of a structure of the smoke detectionapparatus. As shown in FIG. 5, the apparatus 500 includes:

a foreground detecting unit 501 configured to detect foreground blobs inimages of a plurality of frames in a video;

a blob tracking unit 502 configured to determine motion areas ofdetected foreground blobs;

a calculating unit 503 configured to determine a first degree ofoverlapping of the foreground blobs in images of at least two frames,and/or determine a second degree of overlapping of the foreground blobsand motion areas to which the foreground blobs correspond;

a blob removing unit 504 configured to determine interfering foregroundblobs according to the first degree of overlapping and/or the seconddegree of overlapping; and

a smoke detecting unit 505 configured to extract features from motionareas of remaining foreground blobs with the interfering foregroundblobs being removed, and detect smoke according to the features.

In this embodiment, the interfering foreground blobs in the smokedetection are determined and removed according to the degree ofoverlapping of the foreground blobs in images of at least two framesand/or according to the degree of overlapping of the foreground blobsand the motion areas to which the foreground blobs correspond. Hence,the problems existing in the related art may be solved, accuracy ofsmoke detection may be increased, and false detection resulted from suchinterfering objects as water blobs, or light spots, etc., may beavoided.

In this embodiment, reference may be made to operations 101-105 inEmbodiment 1 for implementations of the foreground detecting unit 501,the blob tracking unit 502, the calculating unit 503, the blob removingunit 504 and the smoke detecting unit 505, which shall not be describedherein any further.

In this embodiment, in calculating a first degree of overlapping of aforeground blob, the calculating unit 503 calculates a first overlappingarea of the foreground blob in images of a first number of frames,calculates a first area of the foreground blob, and takes a ratio of thefirst overlapping area to the first area as the first degree ofoverlapping.

For example, the first number is greater than or equal to 2, and thefirst number of frames are a current frame and a frame preceding thecurrent frame in the images of a plurality of frames of the video.Optionally, the first number of frames may be consecutive frames.

In this embodiment, in calculating a second degree of overlapping of aforeground blob, the calculating unit 503 calculates a secondoverlapping area of an area of the foreground blob and a second area ofa motion area of the foreground blob, and takes a ratio of the secondoverlapping area to the second area as the second degree of overlapping.

In this embodiment, the blob removing unit 504 determines a foregroundblob with the first degree of overlapping being greater than or equal toa first threshold as an interfering foreground blob, and/or, the blobremoving unit 504 determines a foreground blob with the second degree ofoverlapping being greater than or equal to a second threshold as aninterfering foreground blob.

For example, the interfering foreground blobs are water blobs and/orlight spots.

With the above embodiment, the interfering foreground blobs in the smokedetection are determined and removed according to the degree ofoverlapping of the foreground blobs in images of at least two framesand/or according to the degree of overlapping of the foreground blobsand the motion areas to which the foreground blobs correspond. Hence,the problems existing in the related art may be solved, accuracy ofsmoke detection may be increased, and false detection resulted from suchinterfering objects as water blobs, or light spots, etc., may beavoided.

Embodiment 3

The embodiment of this disclosure provides an image processing device,including the smoke detection apparatus described in Embodiment 2, withits contents being incorporated herein. The image processing device maybe a computer, a server, a working station, a lap-top computer, and asmart mobile phone, etc.; however, the embodiment of this disclosure isnot limited thereto.

FIG. 6 is a schematic diagram of the image processing device in theembodiment of this disclosure. As shown in FIG. 6, the image processingdevice 600 may include a processor (such as a central processing unit(CPU)) 610 and a memory 620, the memory 620 being coupled to the centralprocessing unit 610. The memory 620 may store various data, andfurthermore, it may store a program for information processing, andexecute the program under control of the processor 610.

In one implementation, the functions of the smoke detection apparatus500 may be integrated into processor 610. The processor 610 may beconfigured to carry out the smoke detection method described inEmbodiment 1.

In another implementation, the smoke detection apparatus 500 and theprocessor 610 may be configured separately. For example, the smokedetection apparatus 500 may be configured as a chip connected to theprocessor 610, with the functions of the smoke detection apparatus 500being carried out under control of the processor 610.

For example, the processor 610 may be configured to perform followingcontrol: detecting foreground blobs in images of a plurality of framesin a video; determining motion areas of detected foreground blobs;determining a first degree of overlapping of the foreground blobs inimages of at least two frames, and/or determining a second degree ofoverlapping of the foreground blobs and motion areas to which theforeground blobs correspond; determining interfering foreground blobsaccording to the first degree of overlapping and/or the second degree ofoverlapping; and extracting features from motion areas of remainingforeground blobs with the interfering foreground blobs being removed,and detecting smoke according to the features.

Reference may be made to Embodiment 1 for a particular implementation ofthe processor 610, which shall not be described herein any further.

Furthermore, as shown in FIG. 6, the image processing device 600 mayinclude a transceiving unit 630, etc. Functions of the above componentsare similar to those in the related art, and shall not be describedherein any further. It should be noted that the image processing device600 does not necessarily include all the parts shown in FIG. 6, andfurthermore, the image processing device 600 may include parts not shownin FIG. 6, and the related art may be referred to.

An embodiment of the present disclosure provides a computer readableprogram, which, when executed in a smoke detection apparatus, will causea computer to carry out the smoke detection method described inEmbodiment 1 in the smoke detection apparatus.

An embodiment of the present disclosure provides a computer readablemedium, including a computer readable program, which will cause acomputer to carry out the smoke detection method described in Embodiment1 in a smoke detection apparatus.

The smoke detection method carried out in the smoke detection apparatusdescribed with reference to the embodiments of this disclosure may bedirectly embodied as hardware, software modules executed by a processor,or a combination thereof. For example, one or more functional blockdiagrams and/or one or more combinations of the functional blockdiagrams shown in FIGS. 5-6 may either correspond to software modules ofprocedures of a computer program, or correspond to hardware modules.Such software modules may respectively correspond to the operationsshown in FIGS. 1 and 4. And the hardware module, for example, may becarried out by firming the soft modules by using a field programmablegate array (FPGA). The method in an apparatus described with referenceto the embodiments of this disclosure may be directly embodied ashardware, software modules executed by a processor, or a combinationthereof. A reference to “unit” as used herein may refer to circuitrystructured as hardware implementation of the smoke detection apparatus.

The soft modules may be located in an RAM, a flash memory, an ROM, anEPROM, and EEPROM, a register, a hard disc, a floppy disc, a CD-ROM, orany memory medium in other forms known in the art. A memory medium maybe coupled to a processor, so that the processor may be able to readinformation from the memory medium, and write information into thememory medium; or the memory medium may be a component of the processor.The processor and the memory medium may be located in an ASIC. The softmodules may be stored in a memory of a smoke detection apparatus, andmay also be stored in a memory card of a pluggable smoke detectionapparatus.

One or more functional blocks and/or one or more combinations of thefunctional blocks in FIGS. 5-6 may be realized as a universal processor,a digital signal processor (DSP), an application-specific integratedcircuit (ASIC), a field programmable gate array (FPGA) or otherprogrammable logic devices, discrete gate or transistor logic devices,discrete hardware component or any appropriate combinations thereofcarrying out the functions described in this application. And the one ormore functional block diagrams and/or one or more combinations of thefunctional block diagrams in FIGS. 5-6 may also be realized as acombination of computing equipment, such as a combination of a DSP and amicroprocessor, multiple microprocessors, one or more microprocessors incommunication combination with a DSP, or any other such configuration.

This disclosure is described above with reference to particularembodiments. However, it should be understood by those skilled in theart that such a description is illustrative only, and not intended tolimit the protection scope of the present disclosure. Various variantsand modifications may be made by those skilled in the art according tothe principle of the present disclosure, and such variants andmodifications fall within the scope of the present disclosure.

For implementations of this disclosure containing the above embodiments,following supplements are further disclosed.

1. A smoke detection apparatus, including:

a foreground detecting unit configured to detect foreground blobs inimages of a plurality of frames in a video;

a blob tracking unit configured to determine motion areas of detectedforeground blobs;

a calculating unit configured to determine a first degree of overlappingof the foreground blobs in images of at least two frames, and/ordetermine a second degree of overlapping of the foreground blobs andmotion areas to which the foreground blobs correspond;

a blob removing unit configured to determine interfering foregroundblobs according to the first degree of overlapping and/or the seconddegree of overlapping; and

a smoke detecting unit configured to extract features from motion areasof remaining foreground blobs with the interfering foreground blobsbeing removed, and detect smoke according to the features.

2. The apparatus according to supplement 1, wherein in calculating afirst degree of overlapping of a foreground blob, the calculating unitcalculates a first overlapping area of the foreground blob in images ofa first number of frames, calculates a first area of the foregroundblob, and takes a ratio of the first overlapping area to the first areaas the first degree of overlapping.

3. The apparatus according to supplement 2, wherein the first number isgreater than or equal to 2.

4. The apparatus according to supplement 2, wherein the first number offrames are a current frame and a frame preceding the current frame inthe images of a plurality of frames of the video.

5. The apparatus according to supplement 2, wherein the first number offrames are consecutive frames.

6. The apparatus according to supplement 1, wherein in calculating asecond degree of overlapping of a foreground blob, the calculating unitcalculates a second overlapping area of an area of the foreground bloband a second area of a motion area of the foreground blob, and takes aratio of the second overlapping area to the second area as the seconddegree of overlapping.

7. The apparatus according to supplement 1, wherein the blob removingunit determines a foreground blob with the first degree of overlappingbeing greater than or equal to a first threshold as an interferingforeground blob, and/or, the blob removing unit determines a foregroundblob with the second degree of overlapping being greater than or equalto a second threshold as an interfering foreground blob.

8. The apparatus according to supplement 1, wherein the interferingforeground blobs are water blobs and/or light spots.

9. A smoke detection method, including:

detecting foreground blobs in images of a plurality of frames in avideo;

determining motion areas of detected foreground blobs;

determining a first degree of overlapping of the foreground blobs inimages of at least two frames, and/or determining a second degree ofoverlapping of the foreground blobs and motion areas to which theforeground blobs correspond;

determining interfering foreground blobs according to the first degreeof overlapping and/or the second degree of overlapping; and

extracting features from motion areas of remaining foreground blobs withthe interfering foreground blobs being removed, and detecting smokeaccording to the features.

10. The method according to supplement 9, wherein in calculating a firstdegree of overlapping of a foreground blob, the method includes:

calculating a first overlapping area of the foreground blob in images ofa first number of frames, calculating a first area of the foregroundblob, and taking a ratio of the first overlapping area to the first areaas the first degree of overlapping;

11. The method according to supplement 10, wherein the first number isgreater than or equal to 2.

12. The method according to supplement 10 or 11, wherein the firstnumber of frames are a current frame and a frame preceding the currentframe in the images of a plurality of frames of the video.

13. The method according to supplement 10 or 11, wherein the firstnumber of frames are consecutive frames.

14. The method according to supplement 9, wherein in calculating asecond degree of overlapping of a foreground blob, the method includes:

calculating a second overlapping area of an area of the foreground bloband a second area of a motion area of the foreground blob, and taking aratio of the second overlapping area to the second area as the seconddegree of overlapping.

15. The method according to supplement 9, wherein the determininginterfering foreground blobs according to the first degree ofoverlapping and/or the second degree of overlapping includes:

determining a foreground blob with the first degree of overlapping beinggreater than or equal to a first threshold as an interfering foregroundblob, and/or, determining a foreground blob with the second degree ofoverlapping being greater than or equal to a second threshold as aninterfering foreground blob.

16. The method according to supplement 9, wherein the interferingforeground blobs are water blobs and/or light spots.

17. An image processing device, including the smoke detection apparatusin supplement 1.

What is claimed is:
 1. An apparatus for smoke detection, comprising: a memory; and a processor coupled to the memory where the processor is configured to: detect foreground blobs in images of a plurality of frames in a video; determine motion areas of the detected foreground blobs; determine a first degree of overlapping of the foreground blobs in images of at least two frames, and/or determine a second degree of overlapping of the foreground blobs and motion areas to which the foreground blobs correspond; determine at least one interfering foreground blob according to the first degree of overlapping and/or the second degree of overlapping; and extract features from motion areas of remaining foreground blobs with the at least one interfering foreground blob being removed, and detect whether smoke exists in the video according to the extracted features, wherein the processor determines a first degree of overlapping of a foreground blob among the foreground blobs, calculates a first overlapping area of the foreground blob in images of a number of frames among the plurality of frames, and calculates a first area of the foreground blob, and takes a ratio of the first overlapping area to the first area as the first degree of overlapping, wherein the number of the frames is greater than or equal to 2, wherein the processor determines a second degree of overlapping of a foreground blob among the foreground blobs, calculates a second overlapping area of an area of the foreground blob and a second area of a motion area of the foreground blob, and takes a ratio of the second overlapping area to the second area as the second degree of overlapping; wherein the processor determines a foreground blob with the first degree of overlapping being greater than or equal to a first threshold as an interfering foreground blob, and/or, determines a foreground blob with the second degree of overlapping being greater than or equal to a second threshold as an interfering foreground blob.
 2. The apparatus according to claim 1, wherein the number of frames using which the processor calculates the first overlapping area include a current frame and a frame preceding the current frame in the images of the plurality of frames of the video.
 3. The apparatus according to claim 1, wherein the number of frames are consecutive frames.
 4. The apparatus according to claim 1, wherein the at least one interfering foreground blob is a water blob and/or a light spot.
 5. A method of smoke detection, comprising: detecting foreground blobs in images of a plurality of frames in a video; determining motion areas of the detected foreground blobs; determining a first degree of overlapping of the foreground blobs in images of at least two frames, and/or determining a second degree of overlapping of the foreground blobs and motion areas to which the foreground blobs correspond; determining at least one interfering foreground blob according to the first degree of overlapping and/or the second degree of overlapping; and extracting features from motion areas of remaining foreground blobs with the at least one interfering foreground blob being removed, and detecting whether smoke exists in the video according to the extracted features, wherein the determining of the first degree of overlapping of the foreground blobs in the images of the at least two frames comprises: in calculating a first degree of overlapping of a foreground blob among the foreground blobs, calculating a first overlapping area of the foreground blob in images of a first number of frames, calculating a first area of the foreground blob, and taking a ratio of the first overlapping area to the first area as the first degree of overlapping, the first number of frames is greater than or equal to 2; and the determining of the second degree of overlapping of the foreground blobs and motion areas to which the foreground blobs correspond comprises: in calculating a second degree of overlapping of a foreground blob among the foreground blobs, calculating a second overlapping area of an area of the foreground blob and a second area of a motion area of the foreground blob, and taking a ratio of the second overlapping area to the second area as the second degree of overlapping, wherein the determining of the at least one interfering foreground blob according to the first degree of overlapping and/or the second degree of overlapping comprises: determining a foreground blob with the first degree of overlapping being greater than or equal to a first threshold as an interfering foreground blob, and/or, determining a foreground blob with the second degree of overlapping being greater than or equal to a second threshold as an interfering foreground blob.
 6. The method according to claim 5, wherein the first number of frames include a current frame and a frame preceding the current frame in the images of the plurality of frames of the video.
 7. The method according to claim 5, wherein the first number of frames include a current frame and a frame preceding the current frame in the images of a plurality of frames of the video.
 8. The method according to claim 5, wherein the first number of frames are consecutive frames.
 9. The method according to claim 5, wherein the first number of frames are consecutive frames.
 10. The method according to claim 5, wherein the at least one interfering foreground blob is water blob and/or light spot.
 11. An image processing device, comprising: a memory that stores a plurality of instructions; and a processor that couples to the memory and configured to execute the instructions to: detect foreground blobs in images of a plurality of frames in a video; determine motion areas of the detected foreground blobs; determine a first degree of overlapping of the foreground blobs in images of at least two frames, and/or determining a second degree of overlapping of the foreground blobs and motion areas to which the foreground blobs correspond; determine at least one interfering foreground blob according to the first degree of overlapping and/or the second degree of overlapping; and extract features from motion areas of remaining foreground blobs with the at least one interfering foreground blob being removed, and detecting whether smoke exists in the video according to the extracted features; wherein the processor determines a first degree of overlapping of a foreground blob among the foreground blobs, calculates a first overlapping area of the foreground blob in images of a number of frames among the plurality of frames, and calculates a first area of the foreground blob, and takes a ratio of the first overlapping area to the first area as the first degree of overlapping; wherein the number of the frames is greater than or equal to 2; wherein the processor determines a second degree of overlapping of a foreground blob among the foreground blobs, calculates a second overlapping area of an area of the foreground blob and a second area of a motion area of the foreground blob, and takes a ratio of the second overlapping area to the second area as the second degree of overlapping; wherein the processor determines a foreground blob with the first degree of overlapping being greater than or equal to a first threshold as an interfering foreground blob, and/or, determines a foreground blob with the second degree of overlapping being greater than or equal to a second threshold as an interfering foreground blob. 